Evolving Features in Neural Networks for System Identification

Given N data pairs {Xi, yi}, i = 1, 2, ..., N , where each Xi is an n-dimensional vector of independent variables (Xi =< xi1 , xi2 , . . . , xin >) and yi is a dependent variable, the function approximation problem (FAP) is finding a function that best explains the N pairs of Xi and yi. From the universal approximation theorem and inherent approximation capabilities proved by various researches, artificial neural networks (ANNs) are considered as powerful function approximators. There are two main issues on the feedforward neural networks’ performance. One is to determine its structure. The other issue is to specify the weights of a network that minimizes its error. Genetic algorithm (GA) is a global search technique and is useful for complex optimization problems. So, it has been considered to have potential to reinforce the performance of neural networks. Many researchers tried to optimize the weights of networks using genetic algorithms alone or combined with the backpropagation algorithm. Others also tried to find a good topology that is even more difficult and called a “black art.” In this paper, instead, we use a genetic algorithm to evolve the input space. That is, a chromosome represents the meanings of input nodes. It generates a new input space using hidden neurons that play a critical role in the learning because they act as feature detectors. They gradually discover the salient features that characterize the training data. We try to find useful input features using a genetic algorithm. A chromosome represents the meaning of the input nodes. A chromosome consists of an array of I functions. Roulette-wheel selection is used for parent selection. A crossover operator creates a new offspring chromosome by choosing some features from two parent chromosomes. Since each neural network has I input nodes and H hidden nodes, there are totally 2(I+H) candidate features to be chosen for offspring. Crossover randomly chooses I features from the set of candidates. This process is shown in Figure 1. The neural network then takes the features in the offspring as the input nodes. The offspring first attempts to replace the inferior out of the two parents. If it fails, it attempts to replace the most inferior member of the population. It stops when there is no improvement during a given number of generations. We attack a critical heat flux (CHF) function approximation problem which is important for the safe and economic design of many heat transfer units including nuclear reactors, fossil-fuel boilers, fusion reactors, electronic chips, etc. Each data set consists of eight independent variables and one dependent variable. We were given 1607 sets of observed data from KAERI (Korea Atomic Energy